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Scheduling  and  Sequence  Reshuffle  for  Autonomous 
Aerial  Refueling  of  Multiple  UAVs 

Zhipu  Jin,  Tal  Shima,  and  Corey  J.  Schumacher 


Abstract —  In  this  paper,  we  formulate  the  autonomous  aerial 
refueling  of  multiple  unmanned  aerial  vehicles  (UAVs)  as  a 
scheduling  problem.  In  order  to  find  the  optimal  refueling 
sequence  of  UAVs,  an  efficient  dynamic  programming  algo¬ 
rithm  is  introduced.  When  UAVs  leave  or  join  the  queue, 
the  optimal  sequence  needs  to  be  recalculated.  A  systematic 
reshuffling  method  is  developed  such  that  the  UAV  sequence 
can  be  reconfigured  by  using  the  least  amount  of  shuffle  steps, 
where  only  one  UAV  changes  its  position  in  each  step.  By 
introducing  a  metric  over  UAV  sequences,  this  reconfiguration 
effort  is  quantified  and  is  treated  as  an  additional  cost  which 
is  integrated  into  the  dynamic  programming  algorithm. 

I.  INTRODUCTION 

Unmanned  Aerial  Vehicles  (UAVs)  are  commonly  used  in 
military  and  civilian  applications.  One  of  the  limitations  of 
many  current  UAVs  is  the  restriction  in  flight  duration  due  to 
the  limited  fuel  capacity.  Having  autonomous  aerial  refueling 
(AAR)  capability  will  allow  UAVs  to  remain  airborne  longer 
and/or  to  take  off  with  a  larger  payload.  Some  work  has 
already  been  done  for  modelling  AAR  docking  maneuvers 
of  a  single  UAV  [1].  In  this  paper,  we  address  the  AAR 
problem  as  a  scheduling  problem  in  which  a  tanker  needs  to 
refuel  multiple  UAVs. 

Considering  the  limited  waiting  time,  finding  the  optimal 
refueling  sequence  for  UAVs  is  similar  to  the  scheduling 
problem  for  a  single  machine  with  “non-resumable”  opera¬ 
tions  [2].  The  problem  is  NP-hard  and  resembles  in  some 
aspects  to  the  restricted  traveling  salesman  problem  with 
time  windows  [3],  [4]  and  to  the  vehicle  routing  problem 
with  time  windows  [5].  Many  efficient  algorithms  have  been 
developed  to  solve  these  problems  such  as  linear  program¬ 
ming,  branch-and-bound,  and  genetic  algorithm.  We  use  the 
dynamic  programming  method  [6],  [7]  to  develop  an  efficient 
algorithm  to  find  the  optimal  sequence.  By  using  a  prior 
examination  and  feasibility  tests  during  the  execution,  the 
proposed  algorithm  reduces  efficiently  the  search  space  in 
cases  where  the  constraints  are  active. 

The  optimal  sequence  needs  to  be  recalculated  whenever  a 
UAV  joins  the  queue  or  leaves  it  unexpectedly.  We  omit  the 
dynamics  of  the  UAVs  and  assume  that  the  UAV  sequence 
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can  be  reconfigured  by  shuffling  the  UAVs’  location.  We 
quantify  the  effort  of  this  reconfiguration  and  integrate  it 
into  the  dynamic  programming  algorithm.  Thus,  optimality 
is  achieved  while  considering  the  reconfiguration  effort. 

The  remainder  of  this  paper  is  organized  as  follows.  In 
section  II,  we  formulate  the  scheduling  problem  for  the 
AAR  of  multiple  UAVs  and  an  efficient  dynamic  program¬ 
ming  algorithm  is  developed.  We  introduce  a  metric  over 
sequences  in  section  III  to  quantify  the  similarities  among 
different  UAV  sequences.  In  section  IV,  three  reshuffling 
algorithms  are  proposed  for  transfering  one  sequence  into 
another.  In  section  V,  the  relationship  between  the  metric  and 
the  number  of  shuffle  steps  is  investigated  and  the  effort  of 
reconfiguration  is  quantified  as  an  additive  cost.  Conclusions 
are  offered  in  section  VI. 

II.  AAR  Scheduling  Problem  of  Multiple  UAVs 

In  this  section,  we  model  the  scheduling  for  the  AAR  of 
multiple  UAVs  as  a  combinatorial  optimization  problem.  A 
tanker  needs  to  provide  refueling  service  for  multiple  UAVs. 
Each  UAV  has  different  parameters  such  as  the  current  fuel 
level,  refueling  time,  and  the  “Retum-to-Field”  priority.  The 
last  parameter,  designated  possibly  by  a  human  operator, 
indicates  how  important  it  is  for  the  UAV  to  return  for 
duty.  The  tanker  gathers  this  information  from  all  UAVs, 
decides  the  optimal  refueling  sequence,  and  sends  the  result 
back  to  each  UAV.  The  UAVs  form  an  echelon  formation, 
as  shown  in  Fig.  1,  following  the  tanker  according  to  the 
optimal  sequence.  We  assume  that  communication  between 
the  tanker  and  the  UAVs  is  ideal,  i.e.,  perfect  information 
is  sent  between  the  tanker  and  UAVs  without  delays  and 
errors.  Thus,  the  problem  we  need  to  solve  is  a  centralized 
optimization  problem. 

A.  Problem  Formulation 

Suppose  that  there  are  N  UAVs  and  each  UAV  is  marked 
by  an  index  i.  The  index  set  is  S  =  {l,---  ,  TV}.  The 
parameters  of  each  UAV  are: 

•  Maximum  waiting  time  wi .  This  parameter  indicates  the 
longest  time  that  the  ith  UAV  can  wait  before  refueling. 
The  value  of  Wi  is  changing  w.r.t.  time,  i.e.  it  reduces 
as  time  progresses.  We  assume  that  >  0  V  i  G  S 
and  that  is  sent,  by  each  UAV,  with  a  time  stamp  to 
enable  synchronization. 

•  Refueling  time  r^.  This  is  the  time  that  the  tanker  needs 
to  fill  up  the  ith  UAV.  It  includes  the  time  of  docking 
maneuvers.  In  order  to  simplify  the  problem,  we  assume 
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Fig.  1.  Echelon  formation  of  UAVs  for  AAR. 


that  Ti  is  time-invariant  and  0  <  r  <  T{  <  r  for  any 

i  g  S. 

•  “Return- to-Field”  priority  pi.  This  positive  number  is 
assigned  to  the  ith  UAV  before  it  is  sent  for  refueling. 
The  bigger  pi  is,  the  higher  the  UAV’s  priority  is. 

•  Refueling  sequence  number  k  G  S.  The  tanker  refuels 
UAVs  according  to  this  number  from  1  to  TV. 

For  any  refueling  sequence,  there  exists  a  bijective  func¬ 
tion  /(•)  :  S  — >  S  such  that  k  =  f(i)  for  any  UAV  i.  The 
cost  function  for  the  AAR  problem  is  defined  as: 


J  = 


N 


i=  1 


Pi 


f(i ) 

k=l 


(1) 


where  rf~1(k)  is  the  total  time  needed  for  refueling 

the  ith  UAV  and  the  UAVs  before  it  in  the  queue.  Suppose 
the  set  of  all  possible  bijective  functions  is  F.  The  optimal 
scheduling  problem  is  finding  the  function  /(•)  G  F  to 
minimize  the  cost  function  J.  We  can  represent  this  as: 


f(-)  =  arg  min  J 

subject  to: 

/(*)-! 

Wi  >  E  rf~1(k )?  Vi  ^  SI. 

k= 1 


(2) 

(3) 


Without  the  time  constraints  of  inequations  (3),  there  are 
totally  TV!  elements  in  F.  However,  the  timing  constrains 
may  make  some  of  them  unfeasible.  Thus,  the  optimal 
scheduling  problem  is  composed  of  two  parts:  (a)  finding 
feasible  sequences,  and  then  (b)  obtaining  the  optimal  one. 
According  to  the  formulation,  the  solution  of  equation  (2) 
is  not  unique.  For  example,  if  two  UAVs  have  the  same 
parameters,  then  they  can  switch  their  position  without 
affecting  the  cost.  In  that  case,  we  do  not  distinguish  between 
these  solutions  and  just  pick  one  heuristically,  e.g.,  that  with 
the  smallest  index  number. 

For  the  sake  of  simplicity,  we  assume  in  this  work  that 
there  always  exists  at  least  one  feasible  solution.  In  an  actual 
implementation  if  a  feasible  solution  will  not  exist  than  it  will 


be  up  to  the  human  operator  to  decide  which  UAV  can  be 
sacrificed,  and  then  the  proposed  algorithm  can  be  re-run. 


B.  Dynamic  Programming  Algorithm 

In  order  to  develop  the  search  algorithm,  a  layered  struc¬ 
ture  with  TV  +  2  layers  of  nodes  is  introduced  in  this  subsec¬ 
tion.  Each  layer  is  marked  by  an  index  j  G  {0, 1,  •  •  •  ,  TV+1} 
which  corresponds  to  one  stage  in  dynamic  programming. 
The  nodes  in  each  layer  represent  UAVs  that  may  be  refueled 
at  that  stage.  We  use  ij  G  S  to  indicate  these  nodes  except 
on  the  initial  layer  (j  =  0)  where  there  is  only  one  virtual 
starting  node  io  =  0  and  on  the  final  layer  (j  =  TV+1)  which 
only  includes  one  sink  node  in+i  =  —  1.  The  nodes  set  on 
each  layer  is  defined  by  Sj  Q  S.  The  scheduling  problem  is 
to  find  an  optimal  path  7r(0,  —  1)  from  the  starting  node  to  the 
sink  node  by  connecting  nodes  on  adjacent  layers.  For  each 
layer,  only  one  node  can  be  visited.  Also,  each  node  can  be 
visited  only  once.  When  the  path  is  found,  the  function  /(•) 
is  determined. 

For  each  layer,  the  node  set  Sj  is  formed  according  to  a 
prior  examination.  For  node  i  G  S',  if  there  exists  a  subset 
K  C.  S\{i}  and  \K\  =  j  —  1  such  that 

^>E Tn’  (4) 

neK 

then  i  G  Sj.  \K\  is  the  number  of  elements  in  the  set  K.  This 
prior  examination  is  important  when  the  time  constraints  are 
tight. 

Following  are  two  lemmas  that  are  easy  to  prove  according 
to  the  above  definition  of  the  layer  structure. 

Lemma  2.1:  If  there  exists  a  feasible  path  in  the  layer 
structure,  then  \Sj\  >  N  +  1  —  j  for  any  j  G  {1,  2,  *  •  •  ,  N}. 

Lemma  2.2:  For  any  j  G  {1,  •  •  •  ,  TV  —  1},  Sj+ 1  C  Sj. 
When  we  construct  the  layer  structure,  we  determine  Sn 
first,  then  find  Sn-i  by  joining  Sn  and  examine  the  results 
over  S\Sn ,  then  find  Sn-2,  and  so  forth.  After  constructing 
the  layer  structure,  we  break  the  problem  into  TV  stages  and 
define  T(0,  —  1)  as  the  cost  of  the  optimal  path  from  the 
initial  node  to  the  sink  node.  The  TV  stages  correspond  to 
the  TV  layers,  excluding  the  initial  and  final  one. 

Before  the  path  reaches  the  sink  node,  it  must  reach  a 
node  %n  E  Sn-  Therefore, 

T(0, -1)=  min  (t(0,  iN)  +  d(iN, -1))  (5) 


where  d(iN,  —  1)  is  the  cost  from  iN  to  the  sink  node  and 
it  equals  PiNTiN.  For  any  other  stage  j ,  given  the  sequence 
{ij,  •  •  •  ,  iiv},  we  have  the  similar  recursion  equation: 


T(0,ij)=  min  (T(0,  ij-i)  +  ij)) 

(6) 

where  the  cost  d(ij-\,ij)  can  be  calculated  by 


N 

d(ij-i,ij)  =  Tij-!  E  Pin-  (7) 

n=j-l 


For  the  initial  layer,  the  recursion  is: 


T(0,*1)=d(0,*1)  (8) 


2 


where  d( 0,  i\)  =  0. 

At  each  stage,  an  additional  feasibility  test  is  needed.  Let 
T  =  J2iLi  Ti  total  refueling  time  for  all  UAVs.  At 

stage  (j  —  1),  a  feasibility  test  for  node  ij- |  G  Sj- 1  \ 
{ij,  •  •  •  ,  ijy}  is  that  if 

N 

wij .,  >  r  L]  Tin,  (9) 

n=j  —  l 

then  is  feasible.  If  there  is  no  node  that  passes  this  test, 
we  let  T(0,  ij )  be  big  enough  such  that  it  cannot  be  selected 
at  the  pervious  stage.  One  reasonable  value  is: 

T(0,ij)  =  N  -  T  •  max(p^).  (10) 

Given  {ij,  •  •  •  ,  ijy},  the  nodes  on  layer  (j  —  1)  that  pass 
the  test  compose  the  feasible  set  i.  When  the  algorithm 
finishes  searching,  if  T(0,  —  1)  >  N  T  ■  ma x(pi),  it  means 
that  there  does  not  exist  a  feasible  refueling  sequence  to  meet 
the  timing  constraints. 

The  computation  complexity  is  sensitive  to  the  timing 
constraints.  In  the  worst  case,  the  timing  constraints  are 
satisfied  by  any  permutation  of  S  and  the  scheduling  problem 
is  solved  in  time  0(N2  2N)  as  discussed  in  [6].  The  easiest 
case  is  when  there  exists  only  one  feasible  sequence  that 
can  meet  the  timing  constraints.  Then,  as  soon  as  the  layer 
structure  is  determined,  the  optimal  sequence  is  found. 

The  dynamic  programming  algorithm  described  above 
can  be  used  to  solve  general  AAR  scheduling  problems. 
Moreover,  according  to  the  structure  of  the  cost  function 
in  equation  (1),  we  have  two  rules  that  greatly  reduce  the 
computation  time. 

Proposition  2.3:  Suppose  at  stage  j  —  1,  flj-i  is  the  fea¬ 
sible  set  of  layer  j  —  l  for  the  given  sequence  {ij,  •  •  •  ,  ijy}. 
For  any  m,  n  G  £X/_ i,  if  rm  =  rn  and  pm  <  pn ,  then 

<  T(0, ij).  (11) 

Proposition  2.4:  Suppose  at  stage  j  —  1,  £lj-i  is  the  fea¬ 
sible  set  of  layer  j  —  1  for  the  given  sequence  {ij,  •  •  •  ,  ijsr}. 
For  any  m,  n  G  Qj- 1,  if  pm  —  Pn  and  rm  <  rn,  then 

T(0,mJ_i)+d(mJ_i,ii)  >  T(0, n^_i)+d(nJ_i, ij).  (12) 

According  to  these  propositions  in  each  recursive  step, 
starting  from  the  end  of  the  queue  and  moving  forward, 
we  pick  the  node  with  the  least  priority  from  those  feasible 
nodes  with  the  same  refueling  time,  or  pick  the  node  with 
the  largest  refueling  time  from  those  with  the  same  priority. 
These  two  propositions  can  reduce  the  complexity  of  the 
scheduling  problem. 

Following  is  an  example  showing  how  the  dynamic  pro¬ 
gramming  algorithm  works.  Suppose  there  are  4  UAVs 
waiting  for  refuelling.  Table  I  lists  all  the  parameters  of  the 
problem  and  Table  II  is  the  layer  structure.  The  nodes  in 
each  column  compose  the  feasible  node  set  Sj .  Those  nodes 
in  bold  form  the  optimal  sequence.  Node  3  is  the  only  one  in 
layer  4,  so  it  must  be  selected.  Then  at  layer  3,  node  1  and  4 
have  the  same  refueling  time.  According  to  Proposition  2.3, 
node  1  is  selected.  After  comparing  the  sequences  {2,4}  and 
{4,2},  we  obtain  the  optimal  sequence  as  {4,2, 1,3}  with 
the  cost  of  98. 


TABLE  I 

Parameters  of  UAVs  in  the  refueling  sequence. 


UVA  index  i 

1 

2 

3 

4 

Max.  waiting  time  Wi 

14 

6 

22 

12 

Refueling  time  ri 

5 

6 

4 

5 

Priorities  pi 

2 

1 

2 

3 

TABLE  II 

Layer  structure  of  initial  scheduling. 


Layer  0 

Layer  1 

Layer  2 

Layer  3 

Layer  4 

Layer  5 

0 

1 

1 

1 

3 

-1 

2 

2 

3 

3 

3 

4 

4 

4 

III.  Similarity  Metric  Between  UAV  Sequences 

For  the  AAR  problem,  the  number  of  UAVs  may  change 
from  time  to  time.  We  assume  that  UAVs  do  not  join  or 
leave  the  queue  simultaneously  and  the  interval  between 
any  two  arrivals  or  departures  is  long  enough  such  that  the 
new  echelon  formation  is  already  formed  before  the  next 
UAV  joins  or  leaves.  In  this  section,  we  focus  on  how  to 
rearrange  the  sequence  when  a  new  UAV  joins.  We  assume 
that,  at  first  step,  the  new  UAV  is  appended  to  the  end  of  the 
echelon  formation.  Then,  after  the  new  optimal  sequence  is 
found,  the  formation  is  reconfigured  accordingly.  Intuitively, 
the  similar  the  new  optimal  sequence  is  to  the  old  one,  the 
less  reconfiguration  is  needed. 

A  metric  is  defined  to  quantify  the  similarity  between 
two  sequences  that  have  the  same  nodes.  Suppose  there  is 
a  node  set  M  which  has  N  nodes.  A  permutation  group 
is  a  sequence  group  G  whose  elements  are  all  permutation 
sequences  of  M.  Any  element  x  G  G  is  a  sequence 
with  N  nodes.  For  each  node  G  M  in  the  sequence 
x  =  [ei,  62,  •  •  •  ,  ejsf\  there  exists  two  adjacent  nodes  (e\  = 
=  e^+i).  For  the  first  node  e\  and  the  last  node 
ejv,  the  adjacent  node  pairs  are  (e[  =  0,e{  =  62)  and 
(elN  =  ejsr-i,erN  =  0),  respectively,  where  0  means 
“None”.  For  any  element  xgG,  the  adjacent  node  pair  of 
node  ei  is  (x(ei)1,  x(ei)r).  For  any  xl,  x2  G  G,  we  assume 
that  the  set  ki  is  composed  of  the  nodes  that  have  identical 
neighbors;  the  set  k 2  is  composed  of  the  nodes  that  only 
have  the  same  left  neighbors;  the  set  k%  is  composed  of  the 
nodes  that  only  have  the  same  right  neighbors;  and  the  set 
&4  is  composed  of  the  nodes  that  have  different  neighbors. 
It  is  clear  that  |Aq|  +  | ^2 1  +  \ks\  +  \k±\  =  N. 

For  a  permutation  group  G  and  node  set  M,  suppose  xl, 
x2  G  G,  a  metric  V{xl,x2)  is  defined  as 

N 

V{xl,x2)  =  £*(«)  (13) 

i= 1 


3 


where  G  M  and 


£(e  i) 


2, 

1, 

1, 

0, 


if  xl(ei)1  x2(ei)1  and  xl(e^)r  7^  x2 (e^)r 
ifxl(e*)z  +  x2(ei)1  andxl(e^)r  =  x2 (e^)r 
ifxl(e^)z  =  x2(ei)1  andxl(e^)r  +  x2(e^)r 
otherwise 


(14) 


For  example,  suppose  M  =  {1,2, 3, 4,  5},  and  G  has  5!  = 
120  elements.  Let  xl  =  [1, 3, 4,  5,  2]  and  x2  =  [3, 4,  5, 1,  2], 
then  72(xl,x2)  =  5  since  both  neighbors  of  node  1  are 
changed,  the  left  neighbors  of  3  and  2  are  changed  and  the 
right  neighbor  of  5  is  changed. 

Fig.  2  shows  the  distribution  of  the  metric  V  for  a 
permutation  group  of  seven  nodes.  We  pick  the  sequence 
[1,  2,  3, 4,  5,  6,  7]  as  the  original  sequence.  The  metric  dis¬ 
tance  V  from  this  origin  to  all  the  other  sequences  in  G  is 
calculated  and  the  Y  axis  represents  the  number  of  sequences 
which  have  the  same  metric  distances  from  the  origin. 


Fig.  2.  Distribution  of  metric 

We  will  now  discuss  some  of  the  general  properties  of 
this  metric.  According  to  the  definition,  22(xl,x2)  =  0  if 
and  only  if  xl  =  x2.  Also,  72(xl,x2)  =  72(x2,xl).  The 
next  lemma  shows  that  V  satisfies  the  triangle  inequality. 

Lemma  3.1:  For  any  xl,  x2,  and  x3  G  G,  D(xl,x2)  + 
72(x2,x3)  >  72(xl,x3) 

Proof:  Suppose  that,  for  xl  and  x2 ,  there  exist  four 
node  sets  as  £q,  /c2,  k%,  and  that  are  defined  before.  So, 

79(xl,  x2)  =  |fc2|  +  |fc3|  +  2|fc4|.  (15) 

For  x2  and  x3 ,  there  exist  the  similar  node  sets  as  &q,  fc2, 
ks,  and  k^.  Thus, 

V(x2,  x3)  =  |fc2|  +  |fc3|  H-  2 1 fc4 1 .  (16) 

Now  suppose  that  the  intersection  of  k\  and  k\  has  r  nodes, 
then  between  xl  and  x3 ,  there  are  at  least  r  nodes  that  have 
identical  neighbors.  We  rewrite  k\  and  k\  as 

f  N  =  r  +  a 

1  |fci|  =  r  +  0.  KU) 


Note  that  these  a  nodes  must  belong  to  /c2  |J  ^3  |J  /C4.  Since 
nodes  in  fc2,  fc3,  or  k 4  make  different  contributions  to  72,  we 
assume  that  there  are  nl  nodes  in  a  that  belong  to  /c2  (J  ^3 
and  n2  nodes  that  belong  to  ^4.  Thus,  we  have 


a  =  nl  +  n2 

72(xl,x2)  =  711  +  2-712+^1. 

For  the  same  reasons,  we  have 


(18) 


/3  =  777 1  +  7772 

V(x2,x3)  =  7771  +  2  •  7772  +  4>2. 

For  xl  and  x3,  we  have 


(19) 


72(xl,  x3) 


<  nl  +  2  •  772  +  777I  +  2  •  7772  +  T+  +  4/2 
=  72(xl,  x2)  +  72(x2,  x3). 

(20) 


Lemma  3.2 :  If  xl,  x2  G  G  and  xl  +  x2,  then  4  < 
72(xl,  x2)  <  27V. 

The  previous  lemma  can  be  proved  easily  according  to  the 
metric  definition. 

IV.  Transfer  between  Sequences 

Transferring  a  sequence  to  another  one,  by  using  efficient 
shuffle  steps,  is  the  main  topic  of  this  section.  The  answer 
directly  affects  the  echelon  formation  sequence  reconfigu¬ 
ration,  whenever  a  UAV  joins  or  unexpectedly  leaves  the 
refueling  queue.  Due  to  the  expected  severe  flight  safety 
requirements  near  the  tanker  we  assume  that  the  reshuffling 
is  performed  for  one  UAV  at  a  time. 

The  single-node  shuffle  is  defined  as: 

Definition  4.1:  A  single-node  shuffle  for  any  element  of 
the  sequence  x  in  the  permutation  group  G,  is  transferring 
one  node  from  it’s  position  in  the  sequence  to  a  different 
one,  while  the  ordering  of  the  other  nodes  is  unchanged. 

For  example,  we  have  a  sequence  with  five  nodes  as 
xl  =  [1,2,  3, 4,  5].  Moving  node  4  to  the  position  between 
1  and  2  results  with  a  new  sequence  x2  =  [1,4, 2, 3,  5]. 
Multiple  single-node  shuffle  steps  may  be  needed  for  a 
sequence  transformation.  Thus,  for  any  xl,  x2  G  G,  an 
efficient  reshuffle  algorithm  generates  a  sequence  of  single¬ 
node  shuffle  steps  such  that,  by  implementing  these  shuffle 
steps,  xl  can  be  transferred  into  x2. 

A.  Reshuffle  algorithm  one 

Suppose  the  initial  sequence  is  xl  =  [ai,  a2,  •  •  •  ,  o/v] 
and  the  finial  sequence  is  x2  =  [&i,&2,  •  •  •  ,  bjsr].  Reshuffle 
algorithm  one  is: 

•  Let  k=l  and  x  =  xl; 

•  From  the  kth  node  in  x,  from  left  to  right,  find  the  node 

ai  in  x  such  that  a*  =  6^.  If  keep  x  and 

directly  jump  to  the  next  step.  Otherwise,  implement  a 
single-node  shuffle  by  moving  a*  to  the  kth  place  and 
generate  a  new  x,  then  go  to  next  step. 

•  Let  k=k+ 1  and  repeat  the  previous  step  until  k  =  N. 

It  is  easy  to  show  that,  in  the  worst  case,  this  algorithm 

needs  TV  —  1  single-node  shuffle  steps  and  TV  (TV  —  l)/2 
comparisons  to  transfer  xl  into  x2.  The  disadvantage  of  this 
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algorithm  is  that  it  cannot  guarantee  to  find  the  minimum 
single-node  shuffle  steps  for  a  sequence  transformation.  For 
example,  suppose  xl  =  [1,  2, 3, 4,  5]  and  x2  =  [2, 3, 4,  5, 1]. 
The  algorithm  needs  four  shuffle  steps  to  transfer  xl  to  x2. 
Obviously,  the  minimum  number  of  single-node  shuffle  step 
is  one  by  moving  node  1  to  the  right  side  of  node  5. 

B.  Reshuffle  algorithm  two 

In  order  to  find  a  better  reshuffle  algorithm,  we  introduce 
the  concept  of  subsequence.  For  xl ,  x2  G  G,  there  exists 
a  subsequence  partition  such  that  each  element  8  in  this 
partition  is  the  largest  non-empty  subsequence  in  which  the 
nodes  keep  the  same  order  for  xl  and  x2.  For  example, 
suppose  xl  =  [1,  2, 3, 4,  5]  and  x2  =  [3, 4,  5, 1,  2].  It  is  easy 
to  see  that  {[1,2],  [3, 4,  5]}  is  the  subsequence  partition  of  xl 
and  x2.  By  switching  the  position  of  these  two  subsequences, 
xl  can  be  transferred  into  x2.  We  define  a  subsequence 
shuffle  as: 

Definition  4.2:  A  subsequence  shuffle  of  sequence  x  is 
moving  a  subsequence  8  to  a  different  location  which  is 
composed  by  |<5|  single-node  shuffles.  The  node  order  inside 
8  is  not  changed. 

The  subsequence  shuffle  is  like  shuffling  a  deck  of  cards 
by  moving  multiple  cards  together.  The  single-node  shuffle  is 
a  specific  case  of  the  subsequence  shuffle.  Thus,  a  sequence 
transformation  can  be  treated  into  two  levels:  subsequence 
level  and  node  level. 

For  a  sequence  transformation,  it  is  important  to  find  the 
subsequence  partition.  Similar  to  the  single  node,  we  define 
the  left  subsequence  neighbor  of  subsequence  8  in  x  as  x(8)1 
and  the  right  subsequence  as  x(8)r.  All  the  elements  of  the 
partition  can  be  put  into  three  subsequence  sets  Ai,  A2,  and 
A3.  Elements  in  Ai  only  have  the  same  left  subsequence 
neighbors  in  xl  and  x2.  Elements  in  A2  only  have  the  same 
right  subsequence  neighbors.  Elements  in  A3  do  not  have 
any  same  subsequence  neighbor  in  xl  and  x2.  Finding  Ai, 
A2,  and  A3  can  be  done  systematically.  Suppose  we  already 
have  ki,  fc2,  &3,  and  k 4  which  are  defined  in  Section  III.  Let 
us  start  from  the  nodes  of  k\.  Suppose  node  a  G  Aq,  then 
subsequence  8  =  [xl(a)z,  e,  xl(a)r]  does  not  change  its  node 
order  in  the  transformation  from  xl  to  x2.  If  xl(e)1  G  Aq, 
then  8  extends  by  adding  xl(e)z’s  left  neighbor  on  its  left 
side  until  this  left  neighbor  is  0.  If  xl(e)z  G  fc3,  then  8 
cannot  extend  itself  on  the  left  side.  The  same  extending 
process  can  be  done  for  xl(e)r.  Eventually,  8  is  extended  to 
be  the  largest  subsequence  including  a.  During  this  process, 
the  nodes  used  to  form  8  are  eliminated  from  Aq,  fc2,  and  fc3. 
When  ki  =  0,  we  check  any  single-node  subsequence  in  fc2. 
If  a  node  b  belongs  to  /c2  and  its  left  neighbor  xl(b)1  7^  0, 
then  xl(b)1  must  belong  to  /c3  since  Aq  =  0.  Thus,  [xl (n)l,n\ 
is  formed  and  belongs  to  A3.  After  the  extending  processes, 
we  can  find  all  the  elements  of  Ai,  A2,  and  A3. 

Lemma  4.3:  |Ai|  <  1  and  |A2|  <  1. 

Proof:  The  only  possible  element  in  A 1  is  the  largest 
subsequence  that  locates  on  the  first  position  from  left  in  xl 
and  x2.  The  same  result  holds  for  A2.  ■ 


Suppose  Ai,  A2,  and  A3  are  subsequences  sets  for  the 
sequence  transformation  from  xl  to  x2.  Reshuffle  algorithm 
two  is: 

•  Find  the  smallest  subsequence  8  in  A3  with  the  condi¬ 
tion  that  no  node  in  8  has  not  been  moved  before. 

•  According  to  x2,  find  the  left  subsequence  x2(8)1  and 
the  right  subsequence  x2(8)r; 

•  Implement  a  subsequence  shuffle  step  such  that 

-  If  x2 (8)1  G  Ai  or  if  x2(8)1  £  Ai  and  \x2(8)l\  > 
\x2(S)r\,  then  put  8  on  the  right  side  of  x2(8)1  to 
form  a  new  subsequence. 

-  If  x2(8)r  G  A2  or  if  x2(8)r  A2  and  |x2(<5)z |  < 
|x2(£)r|,  then  put  8  on  the  left  side  of  x2(8)r  to 
form  a  new  subsequence. 

•  Update  Ai,  A2,  and  A3  according  to  these  new  subse¬ 
quences. 

•  Repeat  the  previous  steps  until  A3  =  0; 

For  reshuffle  algorithm  two,  the  most  important  part  is 
moving  subsequences  in  A3  to  generate  longer  subsequences 
and  to  reduce  the  number  of  elements  of  A3.  Whenever  one 
subsequence  8  is  moved,  the  number  of  elements  of  A3  is 
decreased  at  least  by  one.  This  algorithm  can  be  executed  in 
polynomial  time.  However,  it  still  cannot  guarantee  to  find 
the  minimum  number  of  single-node  steps. 

C.  Reshuffle  algorithm  three 

By  using  the  principle  of  optimality,  we  develop  the 
reshuffle  algorithm  three  to  find  the  minimum  number  of 
single-node  steps. 

Suppose  Ai,  A2,  and  A3  are  subsequences  sets  for  xl  and 
x2  where  A3  has  m  elements.  The  minimum  single-node 
shuffle  steps  we  need  is  represented  by  T{m).  We  have 

T(m)  =  min  ( min(T(ra)z  +  |^|, T(m)r  +  |^|))  (21) 

Sit  a3 

where  Tfm)1  is  the  minimum  single-node  shuffle  steps  we 
need  after  8i  is  moved  to  the  right  side  of  its  left  subsequence 
neighbor,  and  the  same  for  T(m)r.  The  subsequence  set  A3 
needs  to  be  updated  at  each  recursive  step.  Let  m  represent 
the  size  of  A3  after  updating.  Sometimes,  moving  8i  to  the 
right  side  of  its  left  neighbor  is  also  connecting  it  to  its 
right  neighbor.  In  that  case,  these  three  subsequences  are 
formed  into  one  larger  subsequence  and  the  elements  number 
of  A3  is  reduced  by  two.  This  recursive  algorithm  stops  when 
m  =  0. 

By  using  reshuffle  algorithm  three,  T(m)  is  guaranteed 
to  be  the  minimum  number  of  single-node  shuffles.  The 
computation  time  of  this  recursive  algorithm  depends  on  m, 
i.e.,  how  may  elements  exist  in  A3.  In  the  worst  case,  the 
results  are  found  in  time  0(m\). 

D.  Comparison 

In  order  to  verify  the  feasibility  of  these  algorithms,  we 
tested  them  on  permutation  groups  with  different  number 
of  nodes.  The  algorithms  were  coded  in  Matlab  and  run  on 
a  desktop  with  a  Xeon(TM)  CPU  at  2.66  GHz  and  1.00 
GB  of  RAM.  For  each  permutation  group,  we  randomly 
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picked  one  sequence  as  the  initial  one  and  calculated  the 
shuffle  steps  that  transfer  this  initial  sequence  to  all  the  other 
sequences  in  the  permutation  group.  Fig.  3  shows  the  average 
number  of  shuffle  steps  for  the  three  different  algorithms. 
Fig.  4  shows  the  average  computation  time  for  different  algo¬ 
rithms.  According  to  the  simulation  results,  algorithm  three 
guarantees  the  least  amount  of  shuffle  steps  for  a  sequence 
transformation,  but  it  needs  much  more  computation  time  to 
find  solutions  than  the  other  two  algorithms. 


Fig.  3.  Average  number  of  shuffle  steps  for  different  algorithms 


Number  of  nodes 


directly  consider  the  number  of  shuffle  steps  into  the  dynamic 
programming  algorithm  of  section  II.  However,  it  is  intuitive 
that  the  similar  two  sequences  are,  the  less  shuffle  steps  are 
needed.  The  metric  V  defined  in  section  III  can  represent  this 
similarity.  The  relationship  between  the  metric  distance  and 
the  minimum  number  of  shuffle  steps  is  investigated  next. 

We  pick  the  sequence  [1,  2, 3, 4,  5,  6,  7]  as  the  original 
sequence  for  a  permutation  group  of  seven  nodes  and  use 
the  algorithms  from  section  IV.  The  average  values  and 
standard  deviation  of  single-node  shuffle  steps  w.r.t.  the 
value  of  the  metric  V  are  shown  in  Fig.  5  and  Fig.  6.  It 
is  apparent  that,  on  average,  the  bigger  the  metric  distance 
is,  the  more  single-node  shuffle  steps  are  needed.  This  is 
true  for  other  permutation  groups  with  different  number  of 
nodes.  Fig.  7  and  Fig.  8  show  the  results  of  algorithm  three 
for  different  permutation  groups.  Since  the  metric  V  is  an 
additive  function  over  the  nodes,  it  can  be  easily  integrated 
into  the  dynamic  programming  algorithm.  Thus,  we  choose 
the  metric  V  to  indicate  the  cost  of  the  reconfiguration. 


Fig.  5.  Mean  values  of  shuffle  steps  w.r.t.  metric 

Suppose  the  initial  refueling  sequence  is  [A,  •  •  •  ,  Av]-  For 
each  node  ij,  the  two  adjacent  nodes  are  {il-  =  A_iAJ  = 
ij+ 1).  When  a  new  UAV  joins  the  queue,  we  assume  that 
a  new  node  iN+i  is  appended  to  the  end  of  the  queue 
and  7r0  =  [A,-*  -  AivAiV+i]*  The  new  optimal  sequence 
is  indicated  by  7rn.  We  redefine  the  total  cost  function  for 
refueling  scheduling  as: 


Fig.  4.  Average  computation  time  for  different  algorithms 

V.  Sequence  Reconfiguration  for  AAR 

We  assume  that  the  reconfiguration  of  the  UAV  echelon 
formation  is  performed  by  shuffling  the  location  of  one  UAV 
at  each  time.  This  method  can  naturally  ease  the  collision 
avoidance  issue.  The  cost  of  the  reconfiguration  is  related 
to  how  many  shuffle  steps  are  needed;  but  the  shuffle  steps 
can  be  calculated  by  the  algorithms  of  section  IV  only  after 
the  new  sequence  is  determined.  This  means  that  we  cannot 


J  =  (Pi-Etfrf-I{k))+K2v(n0,7rn) 

=  k,  eIV  l Pi  ■  zt(?  *>-(*)) + K ^  eS1  m 
'  '  (22) 
where  /n(-)  is  the  new  scheduling  mapping  function,  the 
second  term  represents  the  metric  distance,  and  (iTi,  K2)  are 
the  weight  coefficients.  Also,  there  are  N+ 1  time  constraints 
listed  below: 

fn(i)~  1 

wi>  Tn1^)’ e  {V-  ,-W  +  !}•  (23) 

k= 1 
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Fig.  6.  Standard  deviation  of  shuffle  steps  w.r.t.  metric 


Fig.  7.  Mean  values  of  shuffle  steps  for  different  permutation  groups 


Fig.  8.  Standard  deviation  of  shuffle  steps  for  different  permutation 
groups 


The  additive  property  of  the  new  cost  function  makes  the 
dynamic  programming  algorithm  in  section  II  still  effective. 
The  cost  in  each  recursive  step  is  calculated  by 

N 

d(ij~i,ij)  =  Ki  ■  Tij_1  ■  ^2  pin  +  K2  ■  £(ij)  (24) 

n=j- 1 

and  =  K2  •  £ (ii). 

VI.  CONCLUSIONS 

In  this  paper,  a  dynamic  programming  algorithm  was 
developed  for  the  AAR  scheduling  problem.  In  this  problem 
one  tanker  needs  to  refuel  multiple  UAVs  flying  in  an  echelon 
formation.  The  optimal  sequence  is  based  on  the  UAVs 
parameters,  including  timing  constraints.  When  refueling 
time  constraints  are  tight,  a  prior  examination  and  feasibility 
tests  in  each  recursive  step  are  necessary  to  reduce  the  search 
space  and  thus  make  the  search  more  efficient. 

When  a  UAV  joins,  or  leaves  unexpectedly,  the  queue,  the 
optimal  sequence  needs  to  be  recalculated.  We  introduced  a 
metric  to  indicate  how  similar  the  new  sequence  is  to  the  old 
one  and  chose  it  as  the  reconfiguration  cost.  The  additive 
property  of  the  metric  makes  it  possible  to  add  it  to  the 
dynamic  programming  algorithm  as  an  additional  cost  term. 
It  was  shown,  for  several  group  sizes,  that  there  exists  a  good 
correspondence  between  this  metric  and  the  number  of  single 
shuffle  steps  needed  for  the  reconfiguration. 

Efficient  algorithms  for  the  reshuffling  have  also  been 
proposed,  including  a  computationally  intensive  one  that 
provides  the  minimum  number  of  shuffle  steps. 
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